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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 



- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 



3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 . 453 O.G. 213. 

Disposition of Claims 

4) [3 Claim(s) 52-110 is/are pending in the application. 

4a) Of the above claim(s) 84-94 and 100-110 is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) 13 Claim(s) 52-83, 95-99 is/are rejected. 
/)□ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 0 The drawing(s) filed on is/are: a)^ accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawlng(s) is objected to. See 37 CFR 1.121 (d). 

1 1) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Status 



1 )^ Responsive to communication(s) filed on 19 April 2004 . 
2a)S This action is FINAL. 2b)n This action is non-final. 



Attach ment(s) 

1) W Notice of References Cited (PTO-892) 

2) CH Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) S Infomnation Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 



4) □ Interview Summary (PTO-413) 



5) en Notice of Infonnrial Patent Application (PTO-152) 

6) □ Other: . 



Paper No(s)/Mail Date. 



Paper No(s)/Mail Date 11. 12. 14 . 



U.S. Patent and Tradennark Office 
PTOL-326 (Rev. 1-04) 
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DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments with respect to claims 52-1 10 have been considered but 
are moot in view of the new ground(s) of rejection. 

2. The applicant argues that the placement zones and trigger zones of Kasier are 
merely temporal portions of a video signal and do not correspond to an "annotation data 
structure element." The examiner disagrees; an annotation data structure element is 
given the broadest reasonable interpretation in the art of: a data structure element 
providing additional information. Therefore, a temporal position is not precluded by the 
given definition of an annotation data structure element. 



Election/Restrictions 

3. Newly submitted claims 84-94 and 100-1 10 are directed to an invention that is 
independent or distinct from the invention originally claimed for the following reasons: 
Newly submitted claims 84-94 and 100-1 10 introduce limitation directed to the encoding 
the packets with transmission techniques employed which is separate and distinct from 
the data structure per se. 

Since applicant has received an action on the merits for the originally presented 
invention, this invention has been constructively elected by original presentation for 
prosecution on the merits. Accordingly, claims 84-94 and 100-1 10 are withdrawn from 
consideration as being directed to a non-elected invention. See 37 CFR 1.142(b) and 
MPEP§ 821.03. 
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Claim Rejections • 35 USC § 101 



4. 



35 U.S.C. 101 reads as follows: 



Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new^ and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 



5. Claims 52, 66, and 95 are rejected under 35 U.S.C. 101 because there is not 
enough functional interrelationship to qualify as a proper data structure. 



6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



7. Claim^^?rafe rejected under 35 U.S.C. 1 03(a) as being unpatentable over U.S. 
Patent 6,615,408 to Kaiser et al. In view of U.S. Patent 5,889,746 to Moriyama et al. 

Regarding claims 52 and 66, Kaiser teaches placement zones (as shown in 
figure 2), which has a reference to an object in the video frame (Abstract), which 
equates to a first annotation data structure element that includes and object reference to 
an object in the video frame. Kaiser teaches a trigger zone which contains a Uniform 
Resource Identifier (URI), which equates to a second annotation data structure element 
referenced by the first identifier and including a set of data references (col. 6, II. 34-39). 
Kaiser teaches the URI being identified by the object in that the URI has a 



Claim Rejections - 35 USC § 103 
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"<videoprod>" field which is not temporally unique into which the trigger is embedded 
(col. 6, II. 43-64. However, Kaiser is silent on the first annotation element identifying the 
second element as claimed. Moriyama teaches identifying elements via pointers, as 
shown in figure 9. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to modify Kaiser by identifying a second element 
via a pointer as taught by Moriyama in order to efficiently manage dynamic resources. 
Further, Kaiser discloses that the image area (described using a placement zone) is 
referenced by some form of an identifier in order to link actions to the image (col. 10, II. 
1-8). 

Regarding claims 53 and 67, Kaiser teaches placing the placement zones in the 
video stream (fig. 2), which reads on a timing data indicator associated with the at least 
on of said plurality of data structure elements. 

Regarding claims 55 and 69, Kaiser teaches placing the placement zones in the 
video stream (fig. 2), which indicates an activation time. 

Regarding claims 56 and 70, Kaiser teaches the placement zones associated 
with a set of video frames (fig. 2, col. 6, II. 18-33), further the trigger is associated with 
the placement zones and consequently is associated with the video program, given the 
broadest reasonable interpretation of associated. 

Regarding claims 57 and 71 , Kaiser is silent on the elements being transmitted 
separately. Official Notice is taken separately sending data in MPEG is well known in 
the art. Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Kaiser by sending the elements separately in 
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order to send data that is larger than the data packets thereby using the existing 
infrastructure to transmit packets. 

Regarding claims 58 and 72, Kaiser teaches a trigger which is used to display 
the action being performed fig. 6A-6b, which reads on the first set of annotation 
references including a data field. However, Kaiser is silent on a second identifier 
referencing a third annotation data structure element. Official Notice is taken that the 
use of pointers is well known in the art. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser to implement 
a pointers (which reads on a reference) in order to access another portion of a data 
structure thereby providing a system that efficiently manages variable length data 
structures and increasing the robustness of the system. 

Regarding claims 59 and 73, Kaiser is silent on the annotation data field is a title 
data field and the third data structure element is a string including the title of the object. 
Moriyama teaches a pointer to a text string wherein the text string can be the title (col. 
15-16, II. 65-31 ; see also fig. 9). Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by pointing 
to a string including the title as taught by Moriyama in order to efficiently manage 
variable length fields and manage the memory of the system. 

Regarding claims 61, 62, 75, and 76, Kaiser is silent on a variable parameter 
field and a variable value. Official Notice is taken that a variable parameter field and 
variable values are well known in the art. Therefore, it would have been obvious to one 
of ordinary skill in the art at the time the invention was made to modify Kaiser by using a 
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variable parameter field and variable values in order to efficiently manage the memory 
by dynamically referencing and allocating the memory. 

Regarding claims 63 and 77, Kaiser is silent on never duplicating the first and 
second identifiers. Official Notice is taken that never duplicating identifiers is well 
known in the art. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to modify Kaiser by never duplicating identifiers 
in order to appropriately reference the appropriate information thereby eliminating 
access to the incorrect data structure. 

Regarding claims 64 and 78, Kaiser teaches location and shape information, 
which equates to a fifth data structure element (fig. 6B, col. 6, II. 20-41). 

Regarding claims 65 and 79, Kaiser teaches that the fifth data structure is 
associated with a video frame (see fig. 2). 

Regarding claims 80 and 96, Kaiser teaches a visual highlight of an image as an 
overlay (see fig. 6B, label 6500, col. 10, II. 20-41 ), which is in the form of an HTML 
table, which equates to a third data structure storing image overlay data, but is silent on 
displaying in response to a user command. Official Notice is taken that customizing 
interfaces based on user preferences is well known in the art. Therefore, it would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
modify Kaiser by customizing the intert'aces in order to control the highlighted portions 
of Kaiser thereby enabling the user to further customize their viewing environment. 

Regarding claims 81 and 97, Kaiser teaches the image overlay data is 
associated with time data and synchronized to the video frames (col. 10, II. 20-41 ). 
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Regarding claims 82 and 98, Kaiser teaclies identifying the object (col. 10, II. 20- 
41 , fig. 6B), but Kaiser is silent on determining the difference from the a first time and 
the current time to initiate the event. Official Notice is taken that determining the 
difference in time to perform an event is well known in the art. Therefore, it would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
modify Kaiser by determining the difference in time to perform an event in order to 
efficiently manage events thereby reducing the processing of the system by selectively 
querying the system clock. 

Regarding claim 95, Kaiser teaches a video production (1100), which reads on a 
first processor for creating the data structures, such as a DVD (col. 5, II. 1-7). Kaiser 
teaches placement zones (as shown in figure 2), which has a reference to an object in 
the video frame (Abstract), which equates to a first annotation data structure element 
that includes and object reference to an object in the video frame. Kaiser teaches a 
visual highlight of an image as an overlay (see fig. 6B, label 6500, col. 10, II. 20-41 ), 
which is in the form of an HTML table, which equates to a third data structure, which 
reads on the third data structure. Kaiser teaches a trigger zone which contains a. 
Uniform Resource Identifier (URI), which equates to a second annotation data structure 
element referenced by the first identifier and including a set of data references (col. 6, II. 
34-39). Kaiser teaches the URI being identified by the object in that the URI has a 
"<videoprod>" field which is not temporally unique into which the trigger is embedded 
(col. 6, II. 43-64. However, Kaiser is silent on the first annotation element identifying the 
second element and third element as claimed. Moriyama teaches identifying elements 
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via pointers, as shown in figure 9. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by 
identifying a second and third element via a pointer as taught by Moriyama in order to 
efficiently manage dynamic resources. Further, Kaiser discloses that the image area 
(described using a placement zone) is referenced by some form of an identifier in order 
to link actions to the image (col. 10, II. 1-8). Kaiser teaches the image overlay data is 
associated with time data and synchronized to the video frames (col. 10, II. 20-41). 

8. Claims 54, 68, 83, and 99 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent 6,615,408 to Kaiser et al. in view of U.S. Patent 
6,415,438 to Blackketter et al. 

Regarding claims 54 and 68, Kaiser is silent on indicating an expiration time. 
Blackketter teaches expiring triggers (col. 3, II. 13-22, col. 10, II. 24-35). Therefore, it 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to modify Kaiser by indicating an expiration time in order to ignore invalid triggers 
(Blackketter: col. 10, II. 24-35). 

Regarding claims 83 and 99, Kaiser is silent on indicating an expiration time. 
Blackketter teaches expiring triggers, which are indicative of the last instance the data 
structure is used (col. 3, II. 13-22, col. 10, II. 24-35). Therefore, it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify 
Kaiser by indicating an expiration time in order to ignore invalid triggers (Blackketter: 
col. 10, II. 24-35). Kaiser and Blacketter are silent on removing the data structures. 
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Official Notice is taken that removing expired information is well known in the art. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Kaiser and Blacketter by removing the data structures in 
order to conserve memory and efficiently manage the memory resources. 

Conclusion 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Andrew Y Koenig whose telephone number is (703) 
306-0399. The examiner can normally be reached on M-Th (7:30 - 6:30). 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Faile can be reached on (703) 305-4380. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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